feat: CI/CD pipeline generation (generate ci / cd / ci-cd)#330
Open
mitanuriel wants to merge 149 commits intodevelopfrom
Open
feat: CI/CD pipeline generation (generate ci / cd / ci-cd)#330mitanuriel wants to merge 149 commits intodevelopfrom
mitanuriel wants to merge 149 commits intodevelopfrom
Conversation
feat(agent): add extended thinking, conversation compaction, and UI i…
…7-14Z chore: release v0.27.0
…2-14Z chore: release v0.27.1
…3-09Z chore: release v0.27.2
feat: updated .gitignore
bug(wrong ref for rig-bedrocks) wrong referenced rig-bedrock package
…3-24Z chore: release v0.28.0
fix: add version to rig-bedrock dependency for crates.io publishing
…8-52Z chore: release v0.28.1
…3-37Z chore: release v0.29.0
…6-17Z chore: release v0.29.1
…5-52Z chore: release v0.29.2
- CloudBuildConfig: steps + optional artifacts.objects + timeout - Every step is a Docker container: name is the image URI (node:20, etc.) - runtime_docker_image() maps Actions identifiers to Docker Hub images - shell_step() uses bash -c for all arbitrary commands (install/lint/test/build) - Docker steps use gcr.io/cloud-builders/docker natively - Image scan: aquasec/trivy image; secret scan: zricethezav/gitleaks image - Artifact upload: top-level artifacts.objects GCS path (no step) - Cache step intentionally skipped: no GCS bucket info at generation time - No trigger block: GCP triggers are console/API only - 23 unit tests; 1391 total passing
- CiConfig: all fields Option<T> + #[serde(default)] so partial configs valid - load_ci_config(): tries .syncable.ci.toml first, then [ci] in .syncable.toml - merge_config_into_context(): applies config layer before CLI flags - parse_platform/parse_format: normalise strings to CiPlatform/CiFormat enums - CiContext: new fields config_test_command, env_prefix, skip_steps, extra_branches - test_helpers: make_minimal_context() alias for CI-22 tests - 15 unit tests; 1406 total passing
…_set, multi-platform sections
- CD-02: CdContext struct + collect_cd_context entry point
- Enums: CdPlatform, DeployTarget, Registry, MigrationTool, Environment
- Detection: Terraform, K8s manifests, Helm charts, migration tools, health checks
- 20 unit tests
- CD-17: CdPipeline schema (platform-agnostic IR)
- Step structs: AuthStep, RegistryStep, DockerBuildPushStep, MigrationStep,
TerraformStep, DeployStep, HealthCheckStep, RollbackInfo, NotificationStep,
EnvironmentConfig
- UnresolvedToken with {{PLACEHOLDER}} formatting
- 17 unit tests
- Token resolver: two-pass resolution engine for CD tokens
- Deterministic pass resolves PROJECT_NAME, IMAGE_NAME, REGISTRY_URL, etc.
- Placeholder pass collects unresolved tokens with deduplication
- 12 unit tests
- CD-22: cd-manifest.toml writer
- Writes [resolved], [unresolved], and [environments] sections
- Supports optional fields with skip_serializing_if
- 7 unit tests
All 58 tests pass. Module wired via cd_generation/mod.rs.
…D-06) - CD-03: registry.rs — ACR, GAR, GHCR login steps + image tag strategy + render_registry_login_yaml() for each registry type + build_image_tag() / build_gar_image_tag() helpers + registry_secrets_doc_entries() for SECRETS_REQUIRED.md + 30 unit tests - CD-04: auth_azure.rs — Azure OIDC auth step (azure/login@v2) + generate_azure_auth() config + render_azure_auth_yaml() + OIDC permissions block helper + 20 unit tests - CD-05: auth_gcp.rs — GCP WIF auth step (google-github-actions/auth@v2) + generate_gcp_auth() config + render_gcp_auth_yaml() + GAR Docker auth helper (gcloud auth configure-docker) + 24 unit tests - CD-06: auth_hetzner.rs — Hetzner SSH + kubeconfig auth + VPS pattern: webfactory/ssh-agent + known_hosts + K8s pattern: kubeconfig write from secret + 28 unit tests All 160 cd_generation tests pass. Zero clippy warnings.
…CD-11) - CD-07 deploy_azure.rs: App Service, AKS, Container Apps deploy generators - CD-08 deploy_gcp.rs: Cloud Run, GKE deploy generators - CD-09 deploy_hetzner.rs: VPS (SSH), HetznerK8s, Coolify deploy generators - CD-10 migration.rs: 7 migration tools (Flyway, Liquibase, Alembic, Django, Prisma, sqlx, Diesel), SSH variant for VPS - CD-11 health_check.rs: curl with retry for HTTP targets, kubectl rollout status for K8s targets - Wire all 5 modules into cd_generation/mod.rs Tests: 290 passed (130 new), 0 failed. Clippy clean.
CD-18: Azure workflow template (templates/azure.rs)
- Full deploy-azure.yml builder from CdPipeline schema
- Renders header, OIDC auth, Docker buildx, deploy steps
(AppService/AKS/ContainerApps), health check, rollback
- 20 tests
CD-19: GCP workflow template (templates/gcp.rs)
- Full deploy-gcp.yml builder from CdPipeline schema
- Renders WIF auth, GAR Docker config, Cloud Run / GKE deploy
- Cloud Run uses step output URL for health check
- 18 tests
CD-20: Hetzner workflow template (templates/hetzner.rs)
- Full deploy-hetzner.yml builder from CdPipeline schema
- GHCR login, SSH agent (VPS/Coolify), kubeconfig (K8s)
- VPS SSH deploy, Coolify webhook, kubectl for HetznerK8s
- 22 tests
CD Writer (writer.rs):
- CdFile/CdFileKind types, write_cd_files(), print_cd_dry_run()
- Conflict detection with force flag, WriteSummary
- 5 tests
Pipeline Builder (pipeline.rs):
- build_cd_pipeline() assembles CdPipeline from CdContext
- Calls auth/registry/deploy/migration/health_check generators
- Maps context Environment to schema EnvironmentConfig
- Default branch filters, namespaces, and replica counts
- 25 tests
CD-01: CLI Entrypoint
- GenerateCommand::Cd variant with --platform, --target,
--registry, --image-name, --dry-run, --output, --force
- CdPlatform, CdTarget, CdRegistry ValueEnum types in cli.rs
- handle_generate_cd() in generate.rs: context → pipeline →
resolve tokens → render template → dry-run or write
- Wired into main.rs, lib.rs, handlers/mod.rs
381 CD tests passing (91 new), 0 failures.
- CD-12 environments.rs: Multi-environment job strategy (13 tests) - CD-13 rollback.rs: Platform-specific rollback scripts (14 tests) - CD-14 reusable_workflow.rs: _deploy-base.yml with workflow_call (14 tests) - CD-15 versioning.rs: Image tag versioning strategy (12 tests) - CD-16 terraform_step.rs: Terraform init/plan/apply steps (15 tests) - CD-21 notification.rs: Slack deployment notifications (18 tests) - CD-29 dispatch.rs: Manual workflow_dispatch inputs (16 tests) - Wire terraform + notification into pipeline builder - Register all 7 new modules in mod.rs - 483 tests passing, 0 clippy warnings
…24/25/26/27/28) CD-23: Combined CI+CD command (generate ci-cd) - Added CiCd variant to GenerateCommand with platform, ci_format, target, registry, image_name, dry_run, output, force, notify flags - handle_generate_cicd orchestrates both generators, cross-links secrets doc - Wired in cli.rs, lib.rs, main.rs with telemetry tracking CD-24: .syncable.cd.toml project-level config - CdConfig struct with all-optional fields (platform, target, environments, registry, image_name, health_check_path, migration_command, default_branch) - load_cd_config: dedicated .syncable.cd.toml takes precedence over shared .syncable.toml [cd] table - merge_config_into_cd_context: config file layer between detection and CLI - 19 unit tests CD-27: CD secrets inventory documentation - collect_cd_secret_names: scans rendered YAML for secrets.* references - secret_metadata: known descriptions for Azure/GCP/Hetzner/Slack secrets - render_cd_secrets_table: markdown table output - generate_cd_secrets_doc: full document generator CD-28: Hetzner prerequisites checklist - Firewall rules (22/80/443/6443), SSH key, Docker, DNS checklist - Auto-appended to secrets doc when platform is Hetzner CD-25: Comprehensive unit tests (33 tests) - cd_snapshot_tests: 24 tests covering Azure/GCP/Hetzner pipeline rendering, environment structure, health check, migration, terraform, notification, rollback, token resolution, multi-platform consistency, no hardcoded secrets - cd_cross_linking_tests: 9 tests covering dispatch/environment consistency, versioning+notification composability, terraform+rollback, reusable workflow CD-26: Integration tests (31 tests) - Full pipeline rendering for all 8 platform/target combinations - No hardcoded secrets validation - Secrets doc generation for Azure/GCP/Hetzner - Context collection from 5 language fixtures (node/python/rust/go/java) - Config loading + merging integration - Cross-platform consistency, health check presence, secrets expression syntax Total: 2,164 tests passing (0 failures)
…view (CD-30) - Added 2c. sync-ctl generate cd: full options table, platform/target matrix, registry defaults, CD steps, multi-env structure, config file (.syncable.cd.toml) docs, output files, and examples - Added 2d. sync-ctl generate ci-cd: options table, all 4 output files, examples for all 3 platforms - Updated table of contents with links to both new sections - Completes CD-30 (Documentation & Help Text for CD Generation)
- Add `migration_command_override: Option<String>` to `CdContext` - `merge_config_into_cd_context()` now sets the field when the config key is present — previously the value was parsed but silently dropped - `build_cd_pipeline()` overrides `MigrationStep.command` with the user-supplied value after tool-derived default is constructed, so the override only applies when a migration tool is also detected - Update all manual `CdContext` fixtures with `migration_command_override: None` - Add tests: merge_migration_command, merge_migration_command_absent_leaves_none, migration_command_override_replaces_tool_default, migration_command_override_without_tool_produces_no_step - README: add `generate cd` and `generate ci-cd` quick-start examples
Rust's line-continuation escape ("\) strips leading whitespace from the
first line of a format string. This caused 4 workflow steps per pipeline
to be emitted at column 0, producing invalid YAML.
Fixed by removing the backslash continuation from all affected format!()
openers in azure.rs (10 sites), gcp.rs (12 sites), hetzner.rs (13 sites).
Also gitignores generated pipeline output files (.github/workflows/deploy-*.yml,
.syncable/cd-manifest.toml, .syncable/SECRETS_REQUIRED.md) so they are
never accidentally committed.
|
Mention Blocks like a regular teammate with your question or request: @blocks review this pull request Run |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Delivers end-to-end CI/CD pipeline generation via three new sync-ctl generate subcommands.
What's included:
generate ci — project-aware CI pipeline (Clippy, tests, Docker Buildx, Trivy SARIF, Gitleaks)
generate cd — platform-specific CD workflows for Azure (app-service, aks, container-apps), GCP (cloud-run, gke), and Hetzner (vps, hetzner-k8s, coolify)
generate ci-cd — single command that generates both in one pass
Unresolved secrets emitted as {{TOKEN}} placeholders with a companion SECRETS_REQUIRED.md
--dry-run flag on all three commands for terminal preview without writing files